Using Constraint Optimization for Conflict Resolution and Detail Control in Activity Recognition

Activity recognition is an open domain which can be used in a wide spectrum of applications, ranging from video surveillance to monitoring and assisting children, elderly or sick people. Some prior work on activity recognition focuses on identifying all occurrences of a specific type of activity. In more general activity recognition settings however (e.g., Ambient Assisted Living), a system should identify all of user activities and their relations, e.g. their temporal sequence or subsumption (for activities hierarchically organized). Due to noise or other imperfections a naïve recognition system may report activities that are logically inconsistent with each other, such as the user is sleeping on the couch and at the same time is watching TV. In this work, we develop a rule-based activity recognition system for hierarchically-organized complex activities that returns only logically consistent sets of activities (scenarios). This is achieved by explicitly formulating activity conflicts as a weighted partial maximum satisfiability problem (Weighted Partial MaxSAT) such that any solution to it corresponds to a set of identified activities (scenario) that do not conflict. The system also has the ability to adjust the desired level of detail of the scenarios returned, e.g., (1 hour TV-watching, 1’ phone-call, 2 hours TV-watching) vs. (3 hours of TV-watching). This is accomplished by assigning preferences to clauses of the Weighted Partial MaxSAT problem. Each complex activity’s weight is calculated by taking into account its confidence, temporal duration and number of used atomic activities. We note that the optimization techniques presented could accommodate other types of preferences and be generalized to other settings. The system is implemented and evaluated in an Ambient Intelligence experimental space.